The Price of Competition: Effect Size Heterogeneity Matters in High Dimensions

In high-dimensional sparse regression, would increasing the signal-to-noise ratio while fixing sparsity level always lead to better model selection? For regression problems, surprisingly, in this paper we answer question negative regime of linear for Lasso method, relying on a new concept term xmlns:xlink="http://www.w3.org/1999/xlink">effect size heterogeneity . Roughly speaking, coefficient vector has high effect heterogeneity if its nonzero entries have significantly different magnitudes. From viewpoint measure, prove that false and true positive rates achieve optimal trade-off xmlns:xlink="http://www.w3.org/1999/xlink">uniformly along path when measure is maximal certain sense, worst achieved it minimal sense all sizes are roughly equal. Moreover, demonstrate first selection occurs much earlier than maximal. The underlying cause these two phenomena is, metaphorically “competition” among variables with same magnitude entering model. Taken together, our findings suggest shall serve as an important complementary coefficients analysis problems. Our proofs use techniques from approximate message passing theory well novel technique estimating rank variable.